The Mozilla Maintenance Service is installed on test machines so that we can test parts of Application Update that rely on it. However, the version installed does not appear to match the version under test.

I verified this by changing the name of the Maintenance Service and then testing what the name of the installed Maintenance Service is: test results, patch. The test reveals that the version installed uses the original Service name, not the updated one.

mhowell: is this situation what you understood to be the case? I'm just wondering if this is historically expected, or a new development.

In any case, it will make development of the MMS almost impossible, and that's not good, so we'll need to get help addressing it. I wonder how the MMS is getting installed on the test machines. Is it possible the test harness itself needs to install or update it?

It looks like the maintenance service gets onto the machines when the images are created, here's an example in a Windows 10 tester, which gets the installer from GitHub, it's installed in the maintenanceservice_install component. (I'm not sure what the separate download of maintenanceservice.exe is for, might be used by another configuration.) These are currently from 27.0a1, some time in 2013-2014.

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #2)

(I'm not sure what the separate download of maintenanceservice.exe is for, might be used by another configuration.)

Answering my own question: maintenanceservice.exe needs to be in the same directory as maintenanceservice_installer.exe for the install to succeed.

(In reply to Nick Alexander :nalexander [he/him] from comment #1)

mhowell: is this situation what you understood to be the case?

No, I'm quite surprised to hear this. I remember having a bad time with getting bug 1643199 to pass tests, and I found about a dozen try pushes for it where I was trying out various things, so I think that must mean try was running the new code and giving me problems. Could it be that what happened in that push was that the GTest job doesn't have the right dependencies set up to get that build's installer to run?

The update tests are able to use the service from the version under test when gServiceTest is set, because it is installed at the end of shouldRunServiceTest(). This copies over the new maintenanceservice.exe into %ProgramFiles(x86)%\Mozilla Maintenance Service.

We might consider doing something similar if we want to extend this to other test suites.

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #5)

The update tests are able to use the service from the version under test when gServiceTest is set, because it is installed at the end of shouldRunServiceTest().

Ah! I didn't realize that. I guess Robert must have known that this was an issue and done that to address it. Unfortunately, the test that I am currently looking into writing targets not just the maintenance service, but the maintenance service installation code. So copying the binary wouldn't be enough to test this. Potentially we could run the installation in a similar way to how we copy the binary now.

It seems to me though that this would quickly become an awful lot of effort to work around a problematic test environment. I was hoping it would be possible for the test environment to have the right version of the test code in the first place.

Folks, thank you for the context and the valuable investigation. I learned some things!

It seems to me though that this would quickly become an awful lot of effort to work around a problematic test environment. I was hoping it would be possible for the test environment to have the right version of the test code in the first place.

Based on the way that our Windows machines are imaged and deployed, it's not feasible to always have the right version deployed out of the box. But there's considerable flexibility to take various steps to set up the tests themselves, and some of that can be shared across test harnesses.

:bytesized, does running the actual maintenance service installer require elevation, or can it be done in automation (say, in an xpcshell test), right now? That would be easier than modifying the (xpcshell, say) test harness.

Otherwise, I suggest we land the patch-du-jour and leave doing better here for follow-up :(

(In reply to Nick Alexander :nalexander [he/him] from comment #7)

:bytesized, does running the actual maintenance service installer require elevation, or can it be done in automation (say, in an xpcshell test), right now? That would be easier than modifying the (xpcshell, say) test harness.

The permissions are set on the test machines so that the maintenance service install dir is world-writeable, but maintenanceservice.exe is still going to need to run as admin to run the service installation tasks relevant for bytesized's tests (I think). The service itself could be used to elevate, but it can really only used to perform an update, so an update would need to be set up.

(In reply to Nick Alexander :nalexander [he/him] from comment #7)

:bytesized, does running the actual maintenance service installer require elevation, or can it be done in automation (say, in an xpcshell test), right now? That would be easier than modifying the (xpcshell, say) test harness.

I think agashlin answered the question pretty well. We either need elevation to run the installer or we could get elevation by basically using the Service to update the Service. It sounds like the latter option is more doable, and would probably suffice for the test I have in mind.